D0CI|HE1IT BESOUE 



ED IBS 756 

AUTHOR 
TITLE 

INSTITUriON 



SPQNS AGENCY 

FEPOFT NO 
PUB DATE 
CONTRACT • 
NOTE 

EDRS .PRLCE 
DESCRIPTORS 

IDENTIFIERS 



. " • EC 123 911 

Deno, 'Stanley I. : And Others 

Experimental ftnalyffis of Proofraw Ccopcnents: An 
Approach to Pesear<i;h in CSDC's, . • 

Minnesota Univ., Minneapolis, Inst, fcr Resea:^;h on V. 
Learninq T)isabili ti>ejs<; 

Bureau of Education for' the Handicapped (DH'EW/OE) 

Washington, D.C. / 

IPLD-BR'12 

Aug 79 • / ■ * . 

300-77-04^1 - / . 

26iJC; See alPO EC 123 901-925. 

« « * 

HF01/PC02. Plus Postaqe. 

*15isabilitiep;: Learnir.a Disabilities; *P^cgraai 
Evaluation : search Methodoloqy 
Pragr am Component T^escarch 



ABSTRACT ' ■ ^ 

'The monoqraph describes an Approach- to Special 
education research that a*ddrep;ses both the {^remise of immediate 
payoff for decision makers i^nherent irf program evaluation /falidat ion , 
and the need to identify effective intervention vafriableg cf model 
programs that could be incorpcrated into other service, settings^ ^The 
approach^ termed ♦'prograiiT componer\t r€search^" is illustrated in a 
successful Child Service Cemonstra+io^n Center for Children with 
Learninq D isabilities • The application of proqrajn component research-- 
demonstrate*:^ that e:»yper iment al research ca^ be integrated within 
existing programs in such a wav^+o benefit, both tetsearc^^ and service. 
(CD ^ ' ^'""^ 



* Veproductions supplied by EDPS '^re'^he best that can 'be made .* 

* from +he oritjinal document. * 



ERIC 



La 



Un University of Minnesota 

, Researcli Report No* 12 



■ DUCATlOWAWiLFAHI * 
NATIONAL INITltUTCOr 
^ 90UCATI0N 

«>OCUMeNT HAS BEEN REPRO- 
ftUCEO EXACTLY AS RECEIVED FROM 

STATED DO NOr.NECeSSARIUV RCPre, 
^nur^i o"^' ^^^'ONAL INSTITUTED, 
EDUCATION PDSITION OR POUCV 



EXPERIMENTAL^NAL'YSIS OF PI^OGRAM COMPONENTs/ " , 
^ ' AN APPROACH TO RESEARCH IN CSDC'S- 

■Stanley Oeno. Berttra™ Chiang. Gerald Tlndal, and Harle Blackburn 




Institute for 
Researcli on 



MAfl FUAI MAS ( N (WtANII 1) hY 



TO WW rtHK^.AfioNAi (^T r,()i;JMcrs 
iNU)nMA!i()N (J Nirn (( lucr" 




Director : James E. Ysseldyke 
Associate. Plrector: PhylllB K. Ml.kln 



' ' xHe In^itJte for Research on Xearnlng "^^f ^Jif/.Li'-f^rre'Hlndl 
• I'^ract (300-77-0J91) With ^ ^ :Lteulre! U.S. Office of 

tapped. Department o Health »ucjtl , institute inves- 

, Education, throuRh Title Vl-G o Public ^^^„,/de,i3lon-maklnB/ 
. ei,ators are conducting ll^^^^f^;; ,TulrTi.^ disabled children. . 
rererrS^r^lfiaerar. or^rnl^ed into el.ht major.areas: . 
1. >de,uacy-of Norm-Referenced Data for prediction 
c# Success 



II.* 



III. 



IV. 



V. 



computer Staulat ion Research on the Assessment/ 

neasion-maSclng/In^'r"''™ ^ 
' u Ar^ rhlldren Labeled LI) tynd 

^^^^^^^ 

surveys 'oUn-tWlncM ' As«essm.nt(Ac fslon Making, 
and Intervention • . X . J 

itholos.lcal Research on Placefe*. Tea| DeclsUJn 

'Milking *».• , ' 



VI ' Bias Following Assessment 

• RcUablllty and Validity of .-ormatlve Kvaluat.on 
- Procedures 

V„l. Data-Utilization Systems In .nstructlonal Pro- 

grarnming 




Additional information on.hese research areas may be obtaioed by «rUln. 
to the Kditor at the Institute. , . , . 



— ~" ' ^'""^ ,.r.ri.,rf under j'ovornmcnt spc/n.sor- 

The ^search- reported herein " f,,, V' tl.el r proPeHslonal 
ship. Contractors ^''/-^''^J^^^'.^ -Mnts of'vlew or oplnlonp 
JudKment In ""^ ,,,'3^" ly cprcsent offU-lal position >of 
•yfnted do not, therefore, nece.sbariiy i 
JhrBureauofWatt^^ ^ 



ERIC 



* « 



J • . 



t 

Refe^arch Repo.rt No. 12 



EXPERIMENT/C^L ANALYSIS OF PROGRAM" COMPONENTS : 
AN APPROACH TO RESEARCH \W CSDC^S 



Stanley Deno Berttram Chlaiig Gerald findal 
University of Minnesota 
Marie BlackJ)urn \ 
Minneapolis Public Schools 
Institute for Resi^arch on Learning Disabilities 
University of Minnesota 



\ 



August, 197.9 



Experimental Analysis .of Program C6mponents 
^ An Approach to Research' In CSDC's: 

The primary purpose of the- program to develop Child Sfervice 



Demonstration Centers for Childreh with Learning Disab^itles (CSDCs) 
(Title VI G) was to provide a system for disseminating programs which 
were demonstr>ably effective. 'From the outset, it could have been argued 
that few, if any, irns truction^l programs existed for which effectiveness 
had been emi^irically demonstrated, and that research rather than develop-- 
ment was what should have been funded. An alternative to this argument • 
was than an empirical 'base for dfBsemlnat ion of a program coul^ be de- 
veloped by requiring CSDCs to undergo a "validation" procedure 
(Tallma.dge, 1977). Validation invol>)^ed collection of evaluation data 
in which the achievement of students -seij^ed through the CSDC>was con- 

trasted with the achievement of students in other educational programs. * 

■f . ' * 

If thfe comparison was carefully done and CSDC students* performance ex- ' 
ceeded that ot compafison students, then program effectiveness was said 
to be empirically "validated" and could legitimately be di3seminate(i. 

While methodological grounds could certainly be found for questioning 
the internal and external validity of individual validation attempts, 
the potential for identifying effective programs through validation cer- 
tainly exists/ Special education programs In general, and mo3el programs 
In particular, should be encouraged Lo continually obtain the best 

11 
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possible data on their ef fectivfeness «inice their very existence is 
predicted on fche. <^ssumption that the additional cost of/such programs ^ 
is offset by increased benefits t/) the students. Program validation, 
however, is in the tradition of ^ficacy 'resear'ch, and adequate ef/icacy 
research is difficult, if not impossible, to *do '(Cegelka & Tyler, 1970)." 
Even when done well, efficacy research is typically based on what Gelfand 
and Hartmann (1975) call "bl6ckbuster ihtefrvention-s. '* Blockbuster - in- 
terventio^s are those in which a complek bundle of treatment variables 
is constructed by the program designers to yield the highest probabil-it^ 

of impacting the problem. * - . 

» . / 

CSDC programs clearly can be classified as blockbuster interventions. 
Coramendably, admijtistrators and teachers do all they can to ^nsufeU+ie^ 
effectiveness of their program. The prbblem .fop potential adopters of 'a 
CSDC program, however, is to know whicji pf all the components ,of the • 
program are really essential to its success. Is it the currfculum, the 
'st^ff competence^ the staff --student ratio, the parent involvement, the 
instructional methodology, or some interaction among all or a subset of 
those ^fomponents' which is Critical to program success? Tht? question^is 
not simply academic, ^ince variations in program c6mponents have economic 

effects. . 

■ ' An alternative or complementary approacli to evaluating blockbuster 
interventions which can simultaneously yield the necessary Information- 
for'aecision makers and increase our knowledge concerning the effective 
variables in treatment is research on prograhi components. 

Propram component research, in contrast to typicn/ program efficacy 

•Ve^.earch, Ls designed to yield data on the effects of varying r-pccific. 
^^omponents within a program rather than on the effects between programs 



• ^ • 

In general. Program component research is much the same as research ^ 

on* instruction in which variations in m^thdchology are empirically 

♦ 

analyzed. It differs in two ^portant 'Respects , however. First, the 

primary goal of program component research is to construct^ a more ef- 

•ficient and effective service progra^. Second, program component re- 

search is typicaUy conducted within the context and constraints of 

existing educational programs. In these two respects, program component 

research is similar to research on instructional theory^ as recently 

outlined by ^now (1977), arid it is clear that research of this type ' 

V ' 

places certain constraints on research design. 

Since the {)rimary purpose of program component research i^t^l^o^ 
struct a better program, and the research is conducted within an exjLstinfg 
program, the selection of treatment contrasts is guided by what variables 
can be ^manipulated within that program. If a program includes ^ 
instruction* in- decoding words, and teachers are free to vary their 
instruction in word decoding, then research can Ke organized to determine 
the effects of such variat^lons. Were decoding instruction either not . 
a program component or a component which could not be manipulated, then . 

TV 

research would not be designed to test the contribution of instraictio«>- 
in decoding to program success. ^ ^ 

While limitiivg research questions to those relevant within a 

■' 

particular instructional program may se^m to confine ,the research in- 
vestigatows, **he truth is that most of the variables of inteFest to 
researchers are present in any program. What tends io te^ limiting is 
how the variables are operationalized by both the staff and the existing 
curriculum. At the same time> the constrainte of program component re- 
search are offset by the direct applicability of the outcomes^ The 
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^'taff q^n use the results to modify practice without 'need ing to ' 
. question, its relevance to theii^.^^rogiram. Then top, if the student- 
population and setting are clearjy ^descri'b.d, other program adminis- 
trators -interested in adopting- a program will, have data on the relative \ 
value of individual program* components. . < " 

To be successful, program component research requires program staff ' 
who are open and inquisiti J about vhat th^y are' doi;;;^. Clearly, such 
'researeh implies that all of what has been included in a program may 
not be necessary or. helpful. Too often special educators are like 
. Campbell's (1969) "trapped administrators," required to acV as if they 
know what. they, are doing 1. correct. Program component research can * 
free program administrators from that trap b.y oper'ating on the premise ' 
•that special education is "experimental education" (Bureilo, fracy, & 
Schultz, 1973), to be used as "developmental capital" (Deno, 1970) for 
improving the capacity of educators to serve individual needs. 

A good illustration of program component research in §pecial edu- 
cation is reported by Jenkins ana Mayhall (1976). At the time of the 
research the authors were involved y developing the Remediation Specialist 
Program through the Southwest Special Education Regional Resource Center. 
In an effort to develop an empirical basis for Remediation Specialist 
functions, Jenkins and his colleagues. experimentally determined the 
effects of such program variables as peer and cross-age tutoring,' tutor- 
pupil ratio, site and supervisor of tutoring, .^nd frequency of tutoring. 
The results of their research -provided them with the d^ta necessary for' 
selecting effective program components, and al^ provided the field of 
special education, tn general. With- guidelines for effective resource ' 
prpgranunlng (Slndelar & Deno, 1977). 
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Program Component Research in- a CSDC , 
In 1975, the Bureau of Education for the Handicapped awarded 
a Title VIG grant to the MiTineapolis Public School^ to develop a 
Child Service Demonstration Center for Children With Learning Disa- 
bilities* Subsequently this proj.ect, eventually knovm as SIMSH^ystem- ^ 
atic T^nstructional Management S^trategies) , became highly successful in 
disseminating its program both regionally and nationally, and underwent 

^ ... ■ ■ ■ 

validation procedures which provided evidence that the program, i-n 
general, was effective. In its final year of funding, the project 
director sought to improve program dissemination by developing a research 
component which might yield data usefdl both to the project anci to the 
^ field of leafnin^ disabilities. The result was .a one-year effort to 
I, conduct a series of studies orf^ the components of the SIMS program. 
Selection of Program Components for Research 

Since a program like SIMS consists of, a large number of interacting ^ 

/ 

components, selecting those components on which to do the research in 
the allotted tim| was a difficult task. As indicated earlier, a balance » 
had to be achieve^! between what program variables might be most important 
to evaluate, and what program variables^^l^jp-d be evaluated within the 
' constraints created by an existing service {Program. The eventual 

selection (ff program components was determined by selecting tjiose which: 
(a) were unique to SIMS, (b) had available on them little research evi- 
dence, (c) could be varied once the school year had begun, (d) were , 
adaptable to a research design enabling sufficient experimental control, 
and (e) could be incorporated in a research design that would be mini- 
mally intrusive with respect to routine staff activities. Using these 
selection criteria, the following program components were included: 



1) The, daily data collection procedures used to monitor 
students* progress on the SIMS reading and spelling skill/ 
concept sequence . 

2) ^ The data utilization techniques based on the daily data 

collection system. * - * 

.3) The instructional techniques used by the teachers in teaching 

the SIMS reading and spelling skills/concepts. ^ 
Specific Research Questions > Just as a service program is a set 
of components, the SIMS components identified f6r research were, in 
» fact, a complex of events for which a number of research questions could 
be generated. The research had to be designed to answer a limited set 
of specific questions with respect>-\:o .each component. Those questions 
were: . * 

1. With respect to the daily data col lectio n procedures , doeh 
graphing student performance data obtained through daily 
timed testing increase rate of mastery beyond what occurs * 
when simply maintaining* the data on recording sheets? 

2. With respect to data utili zation , does -using a decision 
rule based on an. aim line (Liberty, 1975) to make tokens * 
contingent, incre^ase mastery beyond simply setting aim lines 
without decision rules and tokens^ 

3« With respect to instructji onal techniques , docs direct 

instruction in phonetic rudes increase mastery beyond drill 
, -and practice on j^honetically" regular words without instruction 
in rules? 



Method 
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All of the research i was conduoired with the students enrolled in ' 
♦ 

^ the Systematic Instructional Management Strategies (SIMS) Centers at 
the elementary and secondary levels. ^The SIMS Centers provide instruc- 
tion in basic language arts artd math skills to 'those students, gradejf^ 

■' ♦ 

' 1-9, who have not responded to programs offerecj in resource rooms at 
^heir home schools. ^ 

Of the 32 students (18 elementary and 14 secondary) who participated 
in the research, two wexe females. Most of the students had been in the " 

' r 

SIMS program for a period of one to two years. A f^w students did not 

participate in the research either because they ha'd not entered the . 

reading and spelling sequence, or they were progressing too slowly to 

fulfill research design requirements. 

Experimental Design. A within-subject design was used for all 

phases of the- research. The design was created by having each student 
e^erience each treatment , on a different skill/concept category in 

the reading afi& spelling sequence. A student who was on Category #6 
In the sequ4i)te might experience Treatment A on thai>.category, and ' 
Treatment B on Category 7/7. A second student who was in Category m 
might receive Treatment B on that category and Treatment A on Category 
#13. The design was chosen because It introduced experimental controls 
least Intrusive to routine instruction. In addition,' Inter-class'^^ai^d 
inter-suVJecf variability was reduced to a minimum since all subject! 
were assigned to all treatment co.nditions. Treatment'condltlons were- 
assigned to randqnl categories to control for sequence efi^ects and 
the different Icvels-'of category difficulty. 



The experimental research was organized in threeN phases • the 

phases were designed to contrast: (a) graphing daily performance versus 

no graphing of .daily performance, (b), using aimlines and decision rules 

with a token economy versus aimlines and decision rHiles; and (c) drill' 
• '• ■ ' • • . * " 

i ' ' ' 

and pra.ctice versu^ conceptual teaching. An additional 'comparison was 

made across phases between graphing with aimlines and graphing without 
\ . . . ' 

aimlines. * , 

I ' • 

Materials , All of the research was conducted on reading. The 

reading materials consisted of the isolated word li^ts developed by 

the SIMS program.. The words on these lists follow exclusively .the 

« 

word p^itterns for the 53 designated skill/concept categories. 

The other materials used were recording dheets, ^stopwatches, a^d 
semi-logarithmic graph paper for graphing student performance. During 
the second and third phases, the points wh^JJjKerc given^ as token rein- 
forcers could be exchanged primarily for fr^e-timo choice activities, 
and other items such as soft drinks', posters', 'qtc-. In the third phase 
of the study* instructional activities. typically Included the use of 
word and syllable cards. 

Ptocedures. For each experimental phase, a treatment script was , 
developed to outline the specifiic procedures to be followed and 
to ensure consistency Across all subjects. Within each phase,' the orde 
of treatment- conditions was randomly counterbalanced. The standard 
daily timed-test procedure was established using isolated word lists 
associated with each category. A studfent was given, a one-mj^ute ^ 
practice test which was followed by a one-minute Ttraed^est . The pro- 
cedure was essentially the same as used routinely by all the teachers. 
The number of words read correctly and the number'o'f words Incorrect 



from the second timed tei#t were recorded. ^ , ^ 

• jSt'udeVits ordiuai^ll-y pr4cticed Initially on decoding words from % 
*a §jiver\'categor)j without tim^d testing until they could identify .words 

with Sro%.JcbU^cy. A skiH/conce^pt cs^te^Qry was usoc^for t^i^ 
e if a student not done finned, testing, previously on that category,. , 
^tydeh'ts WQrie testeid each "day throCi^hput/the research* Treatments for- 

-a, given category were ajontiaued until a^^;8irudent ^attained a mastery 
I,^,^crit6rian on AQ words per minu**'^^'^ 
^ period of 1^ school days clapa^^, wj*che.ver 'o^C 

the first two phases of the re^fearcl^js was toutine, the daily timed «• 
»t^?ts'-jpf performance on the i|«lated wor3 lists 'v^re admittHgtered. -ex-? ^ 

clusively .by the- instructional a^des. Thet aj/les' c^CHa^^inued the testing 

during Pl^^e ' ir-'but the experimental treatments (variations ' in^ Inetfuct ioa) 

were accomplishedr by the teachers. 



Jt'e^cbrrect on t!wo ^o^ three days, Qr untJLl 
' " * " " ' " ap»6^, wj*che.ver octurred ^irst. •Pd^ing * 



^ Pha^e I (Graphing vs. ^ no graphing): During this first coin^aiJlson * 
30 Students (Ts at Armatage and 12 at^ranklin) had their Ma^ily tljpied'^ 



test score gx^aphed and not graphed. Under the graphing cfonditiotf, the 
resulXs" of l-mlnute timed tests were recorded on a raw data sheet 
J^ephed on Slx-cycjle semi-logarithmic feraph paper. Under tte.no grjiph^ng 
ctJtfdltion, the i^psults ;^.ere. recorded only oh raw data sheets. Under 
both conditions, the^aide^ were instructed to give praise for improved 
•^performance, , , ' k^-;.-. ...v."- 

Phape II (Decision rule and tokens vs. decision r^le ajhdh praise) ! 
For the sec0nd experimental comparison, the. graphing procedures de- 
scribed above-wcre maintained, and a data--„dcclsion rule systeni using 
aim lines Was implemented, to determine. when points for Improvem^t 



10 . 

were to be delivered. For each sutllect, a baseline was established \ 



Be ^t iminMs 



by using the median of three ^timin«s on the first day. TMs data 
point served as a starting po in t;^ f^ 'drawing/ an aim line on the graph- 
A predetermined 30% tn^ease per week ^he baseline was used as 
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the slope for thfc- aim lines. When students were under th^ data-decision* 

rule conditioa, data utilization was accomplished each day by giving. ^ \ 

ome point for reaching the aim line and one extra pointT for every word 
V * , ■ • 

per minute achieved .above the j^m l\ne. *An attempt^ was made to standard- 
ize point values across all classrooms'. A decision rule was used for 
shifting the aim line ^'henevef the subject's performance fell below 'the 
line three days in a row. In this instance, the median level of these 
three dayfe was us^d to reset a parallel aim l*ine (i.'fe., the slope of a 
30% increase remained the same). Each time the aim line was shifted, 

i ^ 

the- number of points provided .for reaching aim line was Increased by 
one. To prevent the subjects from performing poorly in order to^ increase 
point earnings, a 10-point bonus was provided for reaching the aim line 
five days in a row. When stifdents performed under data utilization with 
social consequences the establishment and adjustment of aim lines, was 
the same as previously described. However, only visual and ^verbal feed- 

V • • ' 

back oh performance and praise were provided biT the aldes.^ 

Phase III (Concept teaching vs. drill and practice): Two instruc-- 
tlona^l strategies were contrasted during this phasre - drill and practice 

and what the SIMS staff 'called ''concept t^ching." All previously de- 

> ■ 

Btiribed procedures involving graphing, decision rulcifs, and. points Were 
•maintained during this phase. Eaeh day the teacher provided a ten-rainute 
period of instruction, directed toward the category in which the .timed 



^ • • . . . 11 

• * 

tests were given* The instruction was given prior to testing. The 
drill and practice cgndltion consisted of repetitions of phonetically 

. * ^ . - V- ........ . ^ 

regular words from the n-levant category. The words were presented in \ 
isolation- and In context with error correction, but no direct instruction 
was provided ^n the phortetic (conceptual) rules which underlie category 
^organization. Practice with flash cards, word lists, or T^cope were 
^ the primary modes of^ presentation. Concept teaching, on the other hand, 
focused on direct; instruction in the- phonetic rules underlying' the words. 
Teachers provided direct instruction in word analysis and 3ynthesis 
through verbal explanation and sorting word parts and whole words on 
the basis of the common phonetic rules for decoding words within the 
category . 

R esul ts . \ . M . j} ■ 

a ~ ' ■ V' / 7 

For all three phaaes/the r^uits were analyzed using the number of 
days taken to reach the mastery criterion. Table 1 contains the results 
in t^rms of the mean number of days taken to criterion. The results" 
were stajiist icall^ anal^^zed For the elementary and» socomdary students ^ 
separately using both correlated _t tests and the Wilcoxon tfatched Pair 
Signed Ranks Test. The outcomes for bot\} analyses also are presented in 
Table 1. 

Insert Table 1 about^ here 

Phase I (Graphing vs. no graphing): The analyses w;lth respect to 
graphing produced mixed results. A statistically significant difference 
was obtained using the Wilcoxon Test with the elementfuy student data 
^ but not the junior high data. This result means ^thnt, among the 



I 

elementary students, a significantly higher proportion attained mastery 

<> ' . > 

more rabidly when their dally performifnce was graphed than when it was 

not graphed. , vhien a correlated t test was .conducted on the mean number 

of days required to achieve mastery under each of the treatment condi- 

tions, no difference was obtained at* either the elementary or, secondary 

1* * . 

level. " ^ ' 

Phase TI (Decisiotl rules and points vs- decision rules and •so<:ial 
consequences) t When student "performance under the two decision rule 
condit<lons was examined, significant effects were obtained for both 
mean days to criterion (correlated t) and the proportion of students 
performing better (Wildoxon T) under the data-decision' rule with i^oints 
condition. Again, howe.ver, the effect was^ obtained at elementary level 
but not at the junior high level. 

Phase III (Concept teaching vs. drill and practice): the eg.d . 

. , : ■ ^ -I ^ 

of the school year only four students completed this phase at the 
elementary level, so the majority of the data for this analysis was 
from the junior high level. The Wilcoxon analysis revealed that a 
-higher' proportion of students performed better. under concept teaching 
at the secondary site. Results from the correlated analysis revealed 
no difference in mean nyraber of days.'lo criterion under the. two teaqhing 
conditions. When the data from elementary and secondary sites were 
combined, the same results were obtainlld. 

Cross-Phase Comparison f An atteiApt wan made to evaluate the ef- 
feet of using graphs with. an 'aim'^ line in contrast to graphing without 

I * 

/ 

an aim line. To make this corapai^ison, rfata from the graphing condition 
of Phase I were compared with data from the decision rule and praise ^ 
condition of Phase II. The results of- this cdmpflrlson revealed that a 



higher proportion of students at the elementary site achieved criterion 

faster when daily perfomajice was simply graphed without aim lines* 

No difference was obtained on the mean days -to criterion' under the two 

. • • 

contritions. 

Discussion ' . 

The experimental analysis of SlMS' program <;omponents yielded 
several useful outcomes. One of th'^ most noticeable findings is the 
different effects of the treatment conditions between the two sites, ^ 
A difference in student performance was obtained for each ipaired com- 
parisOn of program components • Three of the effects were obtained/at 
the elementary site and ona at the junior high site • The one signifi- 
cant effect at the jhnior high site occurrefd^ in^ the comparison between 

drill and practice and concept tea'ohlng, while the small number of 

- % 

students completing that phase at the elementary site did not permit • 

an adequate" test of that component for the elementary students, ^he- 
• * ' ♦ ■ * * 

difference between the t«ro sites could be related to several factors. 
One of the most obvious, of course, involves differences in age. Not 
only are students in junior high older than -those in elementary school, 
but they also have experienced^ Ibnger history of reading difficulties 
and intervention attempts. The isolated word lists which, were used 
for both testing and practice often consisted of verfr simple words 
categorized into similar patterns* The older students may have had 
more difficulty persisting on what the'y recognized as a low level reading 
task. Some students at the Junior high site showed a less positive at-- 
titude toward reading the t;ord lists and took part only reluctantj^y . 
This Is, of course, a program component which could be experimentally 



analyzed* 




u 

In addition^ to the differeii#ce In outcomes, for the element^y, 
and junior htgh prografns, several important results were dbtained^' 
First, how the daily measurement data are^ recorded and displayed for 
1$ie students at the e^(?hientary level appeared to be impor.tant« Stu-- 
dents generally attained mastery faster under conditions where they 
and the aides could see daily performance on a graph, A related finding 
was that, for the elementary students^ adding aim lines with decision ^ 
rules for resetting the aijn i^nes' and praise for attainment appeats not 
to Increase the. rate .of achievements that 'conclusion, however, is based 

« * 

on a cottiparison of performance betwe*en categories to which treatments 
were not randomly assigned; tl^erefore, tei%t;^tivGness seems warranted. 
The results, in general, provide support for Including daily graphing 
of student, performance as a compqnent of an intensive-^^ instructional 
program for elementary, aged -learning disabled students. 

' With respect to the. second res^rch question, the reasonable con- 
clusion parallels the firjst outcome. Clear support was obtained for 
the. effectiveness of u$ing a..da ta-decision rule system to deliver tokens 
♦for achievement- Again, |iowever, the effect held with elementary aged 
students rather than the junior high students. The failure to obtain 
an effect for data-det:isi<3n rules with tokens at the junior high level 
is most likely "^iue to either the^particular data-decision rule used or 
the token exchange system and its backup consequences. Since a plethora 
of data exists ^on the effectiveness of tokejn systems, it seems unwar- 
ranted totconclude that the results provide evidence that tiie token^ . 
economy component should not be irifcluded in the junior high program* 
Instead, th|g|^sults provide a ^ffateis for suggesting that more attention 
be given to IdRttlfyliig the appropriate decision rules for the secondary 



students, and 'to determining how the token economy can be made m<ve 
effec'tive. The akswer to the' second research question would seem to 
be that teachers incorporating SIMS into their programs for elementary 
•aged children might well in^tide data-decision rules for delivering 
tokens to increase their effectiveness, 

•Finally, with r^es^ect to 'the relative value of the different ^ 
teaching 'approaches,, the evic^bnce from this analysis provides support 
for direct Instruction in the teachiog of conceptual rules. It, is 
interesting that this conclusion is based on the performance of the 
secondary 'students, slpce propositional logic is something of which 
they are d'#veIopmen tally capable. Unfortunately, the .implication ^' 
that an age by treatment interaction exists is not supportable given 
the small number^of elementary students completing i^ese ^treatments. 
Beyond our program component analysis, the valfie of^ractice in applying 
phonetic rules to decoding is, nevertheless, supported by the results of 
the research. -This finding also is intriguing because the daily timed 
testing procedure' used tended to penalize "sounding out" or verbal 
seU^instrucUons as an aid to performance. Apparently, the effective- 
ness of conceptual teaching accru^ during the teaching time rather 
than the testing time. The results support the recommendation that 
direct teaching of concepts remain a part of the SIMS -program. 

While the results of the experimental research support graphing- 
ot daily perforn^nce;, ysing decision rules for token delivery, and 
direct teaching of c1)ncepts as valuabl\ corapone^its of the SIMS program, 
several caveats should be issued. First, all the research was conducted 
oWomVonents that require a small fraction of the time devoted to 
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reading instruction,^ The proportion of total/variatioj|C in reading 0 
achievement accou^ed forrb.y the^e components cannot f^e ascertained 
through the reaseach that was conducted. A second coa^brn is that 
all the conclusions rest entirely on thermasbery cpit^rion based on 
saying isolated words from category word lists. f * Since the criterion 
validity of this measure is not adequately established, the Importance 

• ' ■ ■/ -^^^ . 

of the results' for o'tliier dimensions of reading performance such as 



comprehensioi\ and silent reading is unknown. 



Summary 



In . this paper we have attempted to describe an appfoach to 
research; in special education that addresses, both the promise of 
Immediate payoff for. decision makers inherent in program evaluation/ 
* validation, and the need to Identify those effective intervention 
variables embedded within model programs that mi^ht be incorporated 
4^to the services delivered by professionals in other settings- We ' ' 
have labeled this approach Vpi^ogram component research," and have pro- 
vided an illustration based on research in one successful Child Servtc6 
Demons;: rat ion Center for Children with Learning Disabilities. The 

research results presented in our illustration ha|ve provided data spe- 

f 

cifically useful to. the decision makers io that program. At the same 
time, however,, the results could be useful to other program administrators 
and to '•researchers interested in the variables Vhich we experimentally 
manipulated, . . 

What may be more Important than the particular research results, 
however, is the demonstration that experimeiital research can be inte- 
grated within existing service programs in a way thft directs and 
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bene*Lts both ^research and service^ The tension experienced by 

r . ^ , i 

federal research funding agencies over what priorl^ty to give proe/am 

development as opposed to •fundamental research njight to some degree 

. V. ' - ' 

be reduced if priority is given «) research /on program components. 



To do so will require a sustained comraitment^f. funds to collaborative 
efforts between research and service institutions. 
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Table 1 . • • . 

Number of Days Taken to Reach the Mastery Criterion. 



7^ 

Wllcoxon 

T 



Site 



[Component 



Phase I 

graphing 

no graphing 

'hase II 

token conseq^uence/ 
decision rule 

social consequence/ 
decision Yule 

?ha8e III 

drill and practice 

concept teaching 

Jrcss-Phase Comparison 

graphing without 
alrallne 

graphing with almllne 



^ N 



X 



Elementary 

correlated Wllcoxon 
SD t T 



N 



Junior High 

correlated 

' SD ■ t 



18 
18 



14 
14 

4 
4 



.72 



11.33 3.82 
11.94 3.84 



\ 

2.28* 



9.36 3.97 

11.00 3.29 

10.25 2.20 

10.00 4.10 



.10 



1.22 



11 '11.36- 3.78 
;i ,12.55 3.56 



22.0** 



13.0** 



5.0 



7.5* 



12 . 9.92 4.62 



12 



14 

13 
13 



12 



^p < .05 



9.92 3.82 



14 6.50 1.99 



7.60 3.36 



7.5/" 2.15 



6.29 1.38 



8.50 4.34 



12 8.75 3.98 

— ^ 



1.01 



1.37 



.23 



17.3^ 



29.0 



9.0* 



27.0 
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